Embedding Heterogeneous Data Using Statistical Models

Embedding algorithms are a method for revealing low dimensional structure in complex data. Most embedding algorithms are designed to handle objects of a single type for which pairwise distances are specified. Here we describe a method for embedding objects of different types (such as authors and terms) into a single common Euclidean space based on their co-occurrence statistics. The joint distributions of the heterogenous objects are modeled as exponentials of squared Euclidean distances in a low-dimensional embedding space. This construction links the problem to convex optimization over positive semidefinite matrices. We quantify the performance of our method on two text datasets, and show that it consistently and significantly outperforms standard methods of statistical correspondence modeling, such as multidimensional scaling and correspondence analysis.